Skip to content

Improve asyncio integration error handling. #4129

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Mar 12, 2025
Merged

Improve asyncio integration error handling. #4129

merged 5 commits into from
Mar 12, 2025

Conversation

antonpirker
Copy link
Member

@antonpirker antonpirker commented Mar 12, 2025 •
edited
Loading

Instrumenting asyncio projects can be confusing. Here are two improvements:

  • If users try to init the Sentry SDK outside of an async loop, a warning message will now printed instructing them how to correctly call init() in async envrionments. Including a link to the docs.

  • During shutdown of Python unfinished async tasks emit an error Task was destroyed but it is pending!. This happens if you use Sentry or not. The error message is confusing and led people to believe the Sentry instrumentation caused this problem. This is now remediated by

    • The tasks is wrapped by Sentry, but we now set the name of the wrapped task to include the original and (and a hint that is has been wrapped by Sentry) to show that the original task is failing, not just some Sentry task unknown to the user.
    • When shutting down a info message is printed, informing that there could be Task was destroyed but it is pending! but that those are OK and not a problem with the users code or Sentry.

Before this PR the users saw this during shutdown:

Exception ignored in: <coroutine object patch_asyncio.<locals>._sentry_task_factory.<locals>._coro_creating_hub_and_span at 0x103ae84f0>
Traceback (most recent call last):
  File "/Users/antonpirker/code/sentry-python/sentry_sdk/integrations/asyncio.py", line 46, in _coro_creating_hub_and_span
    with sentry_sdk.isolation_scope():
  File "/Users/antonpirker/.pyenv/versions/3.12.3/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/Users/antonpirker/code/sentry-python/sentry_sdk/scope.py", line 1732, in isolation_scope
    _current_scope.reset(current_token)
ValueError: <Token var=<ContextVar name='current_scope' default=None at 0x102a87f60> at 0x103b1cfc0> was created in a different Context
Task was destroyed but it is pending!
task: <Task cancelling name='Task-2' coro=<patch_asyncio.<locals>._sentry_task_factory.<locals>._coro_creating_hub_and_span() done, defined at /Users/antonpirker/code/sentry-python/sentry_sdk/integrations/asyncio.py:42> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /Users/antonpirker/.pyenv/versions/3.12.3/lib/python3.12/asyncio/tasks.py:767]>

With this PR the users will see this during shutdown:
Note the INFO message on top and also the task name on the bottom.

[sentry] INFO: AsyncIO is shutting down. If you see 'Task was destroyed but it is pending!' errors with '_task_with_sentry_span_creation', these are normal during shutdown and not a problem with your code or Sentry.
Exception ignored in: <coroutine object patch_asyncio.<locals>._sentry_task_factory.<locals>._task_with_sentry_span_creation at 0x1028fc4f0>
Traceback (most recent call last):
  File "/Users/antonpirker/code/sentry-python/sentry_sdk/integrations/asyncio.py", line 62, in _task_with_sentry_span_creation
    with sentry_sdk.isolation_scope():
  File "/Users/antonpirker/.pyenv/versions/3.12.3/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/Users/antonpirker/code/sentry-python/sentry_sdk/scope.py", line 1732, in isolation_scope
    _current_scope.reset(current_token)
ValueError: <Token var=<ContextVar name='current_scope' default=None at 0x10193ff10> at 0x1029710c0> was created in a different Context
Task was destroyed but it is pending!
task: <Task cancelling name='long_running_task (Sentry-wrapped)' coro=<patch_asyncio.<locals>._sentry_task_factory.<locals>._task_with_sentry_span_creation() done, defined at /Users/antonpirker/code/sentry-python/sentry_sdk/integrations/asyncio.py:58> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[gather.<locals>._done_callback() at /Users/antonpirker/.pyenv/versions/3.12.3/lib/python3.12/asyncio/tasks.py:767]>

Fixes #2908

Improves #2333

@codecov Codecov
Copy link

codecov bot commented Mar 12, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 72.72727% with 6 lines in your changes missing coverage. Please review.

Project coverage is 79.57%. Comparing base (a97c53c) to head (4fe2bcd).
Report is 2 commits behind head on master.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
sentry_sdk/integrations/asyncio.py 72.72% 6 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #4129      +/-   ##
==========================================
+ Coverage   79.53%   79.57%   +0.03%     
==========================================
  Files         140      140              
  Lines       15594    15608      +14     
  Branches     2647     2648       +1     
==========================================
+ Hits        12403    12420      +17     
+ Misses       2351     2349       -2     
+ Partials      840      839       -1
Files with missing lines Coverage Δ
sentry_sdk/integrations/asyncio.py 79.71% <72.72%> (-3.93%) ⬇️

... and 3 files with indirect coverage changes

@antonpirker antonpirker requested a review from sentrivana March 12, 2025 12:42
@antonpirker antonpirker mentioned this pull request Mar 12, 2025
Closed
sentrivana
sentrivana approved these changes Mar 12, 2025
View reviewed changes
Copy link
Contributor

@sentrivana sentrivana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is good. We might consider changing the log level of the shutdown_handler message to debug, but it's probably also fine as info since it should only be logged at shutdown.

@antonpirker
Copy link
Member Author

I think info is fine, because it is only at shutdown and it will then be visible in more user projects out of the box.

@antonpirker antonpirker merged commit 50b1919 into master Mar 12, 2025
141 of 142 checks passed
@antonpirker antonpirker deleted the antonpirker/fix/asyncio-shot-down branch March 12, 2025 14:12
@kerenkhatiwada kerenkhatiwada mentioned this pull request Apr 2, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sentrivana sentrivana sentrivana approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Task was destroyed but it is pending: _coro_creating_hub_and_span
2 participants
@antonpirker @sentrivana